Intraoral scanner with touch sensitive input

ABSTRACT

A method includes: displaying to a display a first indication of a first segment of a patient&#39;s oral cavity to be scanned; receiving a first user input via a hand held intraoral scanning device; causing the hand held intraoral scanning device to enter a scan mode; receiving one or more intraoral scans of the first segment; generating a three-dimensional (3D) model of the first segment based on the one or more intraoral scans; displaying to the display the 3D model of the first segment; receiving a second user input via the intraoral scanning device after receiving the one or more intraoral scans of the first segment; and displaying to the display an overlay including a first element corresponding to third user input for navigating to a next segment and a second element corresponding to fourth user input for navigating to a previous segment.

RELATED APPLICATION

This application is a continuation application of U.S. patentapplication Ser. No. 14/641,188, filed Mar. 6, 2015, the entire contentsof which are hereby incorporated by reference herein.

TECHNICAL FIELD

Embodiments of the present invention relate to the field of intraoralscanning and, in particular, to an intraoral scanner with touchsensitive input.

BACKGROUND

In prosthodontic procedures designed to implant a dental prosthesis inthe oral cavity, the dental site at which the prosthesis is to beimplanted in many cases should be measured accurately and studiedcarefully, so that a prosthesis such as a crown, denture or bridge, forexample, can be properly designed and dimensioned to fit in place. Agood fit enables mechanical stresses to be properly transmitted betweenthe prosthesis and the jaw, and to prevent infection of the gums via theinterface between the prosthesis and the dental site, for example.

Some procedures also call for removable prosthetics to be fabricated toreplace one or more missing teeth, such as a partial or full denture, inwhich case the surface contours of the areas where the teeth are missingneed to be reproduced accurately so that the resulting prosthetic fitsover the edentulous region with even pressure on the soft tissues.

In some practices, the dental site is prepared by a dental practitioner,and a positive physical model of the dental site is constructed usingknown methods. Alternatively, the dental site may be scanned to provide3D data of the dental site. In either case, the virtual or real model ofthe dental site is sent to the dental lab, which manufactures theprosthesis based on the model. However, if the model is deficient orundefined in certain areas, or if the preparation was not optimallyconfigured for receiving the prosthesis, the design of the prosthesismay be less than optimal. For example, if the insertion path implied bythe preparation for a closely-fitting coping would result in theprosthesis colliding with adjacent teeth, the coping geometry has to bealtered to avoid the collision, which may result in the coping designbeing less optimal. Further, if the area of the preparation containing afinish line lacks definition, it may not be possible to properlydetermine the finish line and thus the lower edge of the coping may notbe properly designed. Indeed, in some circumstances, the model isrejected and the dental practitioner then re-scans the dental site, orreworks the preparation, so that a suitable prosthesis may be produced.

In orthodontic procedures it can be important to provide a model of oneor both jaws. Where such orthodontic procedures are designed virtually,a virtual model of the oral cavity is also beneficial. Such a virtualmodel may be obtained by scanning the oral cavity directly, or byproducing a physical model of the dentition, and then scanning the modelwith a suitable scanner.

Thus, in both prosthodontic and orthodontic procedures, obtaining athree-dimensional (3D) model of a dental site in the oral cavity is aninitial procedure that is performed. When the 3D model is a virtualmodel, the more complete and accurate the scans of the dental site are,the higher the quality of the virtual model, and thus the greater theability to design an optimal prosthesis or orthodontic treatmentappliance(s).

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the present invention are illustrated by way of example,and not by way of limitation, in the figures of the accompanyingdrawings.

FIG. 1 illustrates one embodiment of a system for performing intraoralscanning and generating a virtual three dimensional model of a dentalsite.

FIG. 2 illustrates a perspective view of an intraoral scanner with touchsensitive input.

FIG. 3A illustrates a flow diagram for a method of receiving a touchinput from a touch sensitive intraoral scanner and performing adetermined function during an intraoral scan session, in accordance withembodiments of the present invention.

FIG. 3B illustrates a flow diagram for a method of enabling anddisabling a touch sensor of an intraoral scanner and performingfunctions based on inputs from the touch sensor, in accordance withembodiments of the present invention.

FIG. 3C illustrates a flow diagram for a method of generating medicalimages, providing the medical images to a computing device, and thenmanipulating those medical images on the computing device, in accordancewith embodiments of the present invention.

FIG. 4 illustrates an example scan mode user interface of an intraoralscan application.

FIG. 5 illustrates an example rotation of an intraoral scanner and theresulting rotation of a virtual 3D model.

FIG. 6 illustrates example swipe gestures and resulting functionsperformed by an intraoral scan application.

FIG. 7 illustrates an example overlay screen of a intraoral scanapplication.

FIG. 8 illustrates a block diagram of an example computing device, inaccordance with embodiments of the present invention.

DETAILED DESCRIPTION

Described herein is a method and apparatus for improving medicalscanning using a touch sensitive medical scanning device, such as atouch sensitive intraoral scanner. During a scan session, a user (e.g.,a dental practitioner) of a scanner may generate multiple differentimages (also referred to as scans or medical images) of a dental site,model of a dental site, or other object. The images may be discreteimages (e.g., point-and-shoot images) or frames from a video (e.g., acontinuous scan). Existing medical scanning solutions frequently involvethe user holding the scanner to engage the patient for scanning,disengaging from the patient to address a medical scan applicationexecuting on a computing device, then reengaging with the patient tocontinue the scanning process, again disengaging from the patient toaddress the medical scan application, and repeating until completion ofa scanning session. Such processes can be quite cumbersome andinefficient. Moreover, medical scanning devices generally lack theability to both generate medical images and then manipulate thosemedical images or representations thereof on a display of a computingdevice.

Embodiments of the present invention enable a user to perform operations(such as to control or navigate a user interface and/or to manipulatemedical images or a representation generated from medical images) whilestill engaged with a patient that in previous systems could only beperformed by disengaging from the patient and interacting with acomputing device running an intraoral scan application. The ability toperform such operations while still engaged with the patient can improvethe efficiency of a workflow for scanning a patient.

In one embodiment, a computing device executing a medical scanapplication receives a touch input from a touch sensitive medicalscanning device during a medical scan session. The medical scanningdevice may be, for example, an intraoral scanner that includes a touchsensor (e.g., a touch pad). The computing device determines whether thetouch input is a hold gesture or a swipe gesture. The computing devicethen performs a first function or operation to control a user interfaceof the medical scan application if the touch input is a hold gesture anda second function or operation to control the user interface of themedical scan application if the touch input is a swipe gesture. Examplesof functions that may be performed include activating a gyroscope in themedical scanning device, using data from the gyroscope to control anorientation of a virtual 3D model (e.g., if a hold gesture is detected)and proceeding to next or previous scan segments (e.g., if a swipegesture is detected). The functions or operations performed responsiveto the hold or swipe gestures may be functions that traditionally areperformed responsive to a user using a keyboard, mouse and/or touchscreen of the computing device. By providing touch sensors in themedical scanning device and a medical scan application that can respondto touch input from such touch sensors, embodiments improve theefficiency of performing medical scans.

In one embodiment, a medical scanning device includes an image sensor, acommunication module and a touch sensor. The image sensor generatesmedical images of a patient and the communication module transmits thosemedical images to a computing device, which may then display the medicalimages or a representation of the patient generated from the medicalimages (e.g., a 3D virtual model of a dental site of the patient). Thetouch sensor is then activated (e.g., by a button push combination ofone or more buttons), and a user then uses the touch sensor tomanipulate the medical images or the representation generated from themedical images. Thus, the same medical scanning device may be used bothfor generation of the medical images and manipulation of the medicalimages and a user interface that receives and operates on the medicalimages.

Embodiments described herein are discussed with reference to intraoralscanners, intraoral images, intraoral scan sessions, intraoral scanapplications, and so forth. However, it should be understood thatembodiments also apply to other types of scanners than intraoralscanners. Embodiments may apply to any type of medical scanning device,such as those that take multiple images and stitch these images togetherto form a combined image or virtual model. For example, embodiments mayapply to desktop model scanners, computed tomography (CT) scanners, andso forth. Embodiments may also apply to ultrasound devices that includeultrasound transceivers, x-ray devices that include an x-ray emitterand/or an x-ray detector, and other devices. Additionally, it should beunderstood that the intraoral scanners or other scanners may be used toscan objects other than dental sites in an oral cavity. For example,embodiments may apply to scans performed on physical models of a dentalsite or any other object. Accordingly, embodiments describing intraoralimages should be understood as being generally applicable to any typesof images generated by a scanner, embodiments describing intraoral scansessions should be understood as being applicable to scan sessions forany type of object, embodiments describing intraoral scan applicationsshould be understood as being applicable to medical scan applications,and embodiments describing intraoral scanners should be understood asbeing generally applicable to many types of scanners.

FIG. 1 illustrates one embodiment of a system 100 for performingintraoral scanning and/or generating a virtual three dimensional modelof a dental site. In one embodiment, system 100 carries out one or moreoperations below described in methods 300 and 350. System 100 includes acomputing device 105 that may be coupled to a touch sensitive scanner150 and/or a data store 110.

Computing device 105 may include a processing device, memory, secondarystorage, one or more input devices (e.g., such as a keyboard, mouse,tablet, touch screen, and so on), one or more output devices (e.g., adisplay, a printer, etc.), and/or other hardware components. Computingdevice 105 may be connected to a data store 110 either directly or via anetwork. The network may be a local area network (LAN), a public widearea network (WAN) (e.g., the Internet), a private WAN (e.g., anintranet), or a combination thereof. The computing device 105 may beintegrated into the scanner 150 in some embodiments to improveperformance and mobility.

Data store 110 may be an internal data store, or an external data storethat is connected to computing device 105 directly or via a network.Examples of network data stores include a storage area network (SAN), anetwork attached storage (NAS), and a storage service provided by acloud computing service provider. Data store 110 may include a filesystem, a database, or other data storage arrangement.

In some embodiments, a touch sensitive scanner 150 for obtainingthree-dimensional (3D) data of a dental site in a patient's oral cavityis operatively connected to the computing device 105 via a communicationmodule of the touch sensitive scanner 150. The computing device 105 maybe connected to the touch sensitive scanner 150 directly or indirectlyand via a wired or wireless connection. For example, the touch sensitivescanner 150 may include a communication module such as a networkinterface controller (NIC) capable of communicating via Wi-Fi, via thirdgeneration (3G) or fourth generation (4G) telecommunications protocols(e.g., global system for mobile communications (GSM), long termevolution (LTE), Wi-Max, code division multiple access (CDMA), etc.),via Bluetooth, via Zigbee, or via other wireless protocols.Alternatively, or additionally, touch sensitive scanner 150 may includean Ethernet network interface controller (NIC), a universal serial bus(USB) port, or other wired port. The NIC or port may connect the touchsensitive scanner to the computing device 105 via a local area network(LAN). Alternatively, the touch sensitive scanner 150 may connect to awide area network (WAN) such as the Internet, and may connect to thecomputing device 105 via the WAN. In an alternative embodiment, touchsensitive scanner 150 is connected directly to the computing device 105(e.g., via a direct wired or wireless connection). In one embodiment,the computing device 105 is a component of the touch sensitive scanner150.

Touch sensitive scanner 150 may include a probe (e.g., a hand heldprobe) for optically capturing three dimensional structures (e.g., byconfocal focusing of an array of light beams). Touch sensitive scanner150 additionally includes one or more touch sensors (e.g., one or moretouch pads) that can receive touch input. Touch sensitive scanner 150may also include other components such as optical components, anaccelerometer, communication components, a gyroscope, processingdevices, and so on. One example of a touch sensitive scanner 150 is theiTero® intraoral digital scanner manufactured by Align Technology, Inc.

The touch sensitive scanner 150 may be used to perform an intraoral scanof a patient's oral cavity. An intraoral scan application 108 running oncomputing device 105 may communicate with the touch sensitive scanner150 to effectuate the intraoral scan. A result of the intraoral scan maybe a sequence of intraoral images that have been discretely generated(e.g., by pressing on a “generate image” button of the scanner for eachimage). Alternatively, a result of the intraoral scan may be one or morevideos of the patient's oral cavity. An operator may start recording thevideo with the touch sensitive scanner 150 at a first position in theoral cavity, move the touch sensitive scanner 150 within the oral cavityto a second position while the video is being taken, and then stoprecording the video. In some embodiments, recording may startautomatically as the scanner identifies teeth and/or other objects. Thetouch sensitive scanner 150 may transmit the discrete intraoral imagesor intraoral video (referred to collectively as intraoral image data135) to the computing device 105. Computing device 105 may store theimage data 135 in data store 110. Alternatively, touch sensitive scanner150 may be connected to another system that stores the image data indata store 110. In such an embodiment, touch sensitive scanner 150 maynot be directly connected to computing device 105.

According to an example, a user (e.g., a practitioner) may subject apatient to intraoral scanning. In doing so, the user may apply touchsensitive scanner 150 to one or more patient intraoral locations. Thescanning may be divided into one or more segments. As an example, thesegments may include a lower buccal region of the patient, a lowerlingual region of the patient, a upper buccal region of the patient, anupper lingual region of the patient, one or more preparation teeth ofthe patient (e.g., teeth of the patient to which a dental device such asa crown or an orthodontic alignment device will be applied), one or moreteeth which are contacts of preparation teeth (e.g., teeth notthemselves subject to a dental device but which are located next to oneor more such teeth or which interface with one or more such teeth uponmouth closure), and/or patient bite (e.g., scanning performed withclosure of the patient's mouth with the intraoral scanner being directedtowards an interface area of the patient's upper and lower teeth). Viasuch scanner application, the touch sensitive scanner 150 may provideintraoral image data (also referred to as scan data) 135 to computingdevice 105. The intraoral image data 135 may include 2D intraoral imagesand/or 3D intraoral images. Such images may be provided from the scannerto the computing device 105 in the form of one or more points (e.g., oneor more pixels and/or groups of pixels). For instance, the touchsensitive scanner 150 may provide such a 3D image as one or more pointclouds.

The manner in which the oral cavity of a patient is to be scanned maydepend on the procedure to be applied thereto. For example, if an upperor lower denture is to be created, then a full scan of the mandibular ormaxillary edentulous arches may be performed. In contrast, if a bridgeis to be created, then just a portion of a total arch may be scannedwhich includes an edentulous region, the neighboring abutment teeth andthe opposing arch and dentition. Thus, the dental practitioner may inputthe identity of a procedure to be performed into medical scanapplication 108. For this purpose, the dental practitioner may choosethe procedure from a number of preset options on a drop-down menu or thelike, from icons or via any other suitable graphical input interface.Alternatively, the identity of the procedure may be input in any othersuitable way, for example by means of preset code, notation or any othersuitable manner, medical scan application 108 having been suitablyprogrammed to recognize the choice made by the user. In either case, themedical practitioner may generate a treatment plan that includes one ormore segments that are to be scanned. A segment (or scan segment) mayinclude a particular tooth (e.g., a preparation tooth), an upper orlower arch, a portion of an upper or lower arch, a bite, and so on.

By way of non-limiting example, dental procedures may be broadly dividedinto prosthodontic (restorative) and orthodontic procedures, and thenfurther subdivided into specific forms of these procedures.Additionally, dental procedures may include identification and treatmentof gum disease, sleep apnea, and intraoral conditions. The termprosthodontic procedure refers, inter alia, to any procedure involvingthe oral cavity and directed to the design, manufacture or installationof a dental prosthesis at a dental site within the oral cavity, or areal or virtual model thereof, or directed to the design and preparationof the dental site to receive such a prosthesis. A prosthesis mayinclude any restoration such as crowns, veneers, inlays, onlays, andbridges, for example, and any other artificial partial or completedenture. The term orthodontic procedure refers, inter alia, to anyprocedure involving the oral cavity and directed to the design,manufacture or installation of orthodontic elements at a dental sitewithin the oral cavity, or a real or virtual model thereof, or directedto the design and preparation of the dental site to receive suchorthodontic elements. These elements may be appliances including but notlimited to brackets and wires, retainers, clear aligners, or functionalappliances.

A user (e.g., a practitioner) may navigate through scanning segments viaa user interface (UI) of the intraoral scan application 108 by variousinput devices, such as a cursor control device (e.g., a mouse) or atouch input device (e.g., touchscreen). To use such means in somesystems, the user typically disengages from the patient to engage thecomputing device 105 to operate the intraoral scan application 108. Forexample, a treatment plan may indicate that an upper arch region, lowerarch region, and bite region of a patient are to be scanned.Traditionally, the user navigates the user interface to prepare theintraoral scan application 108 for the scanning of the upper arch regionusing a touchscreen or mouse device. The user then moves back to thepatient to perform a scan for the upper arch region. The user then movesto the computing device to navigate to the next segment to be scanned.The user once again moves back to the patient to perform the nextsegment scan. The user repeats the process until all segments areproperly scanned.

In embodiments, a touch sensitive scanner 150 may allow the user toeasily navigate or control the user interface of the intraoral scanapplication 108 using the touch input of the touch sensitive scanner150, thereby minimizing instances of the user moving between thecomputing device and the patient. For example, the user may utilize acombination of buttons and various touch gestures on the touch sensor ofthe touch sensitive scanner 150 to navigate the intraoral scanapplication 108 without moving to the computing device 105 to navigateor control the user interface.

Intraoral scan application 108 may include various modules to facilitateintraoral scanning procedures. In one embodiment, intraoral scanapplication 108 includes a touch input module 122, a patient module 115,a scan module 118, an image processing module 125, and a delivery module120. Intraoral scan application 108 may additionally include a userinterface 112, such as a graphical user interface (GUI). In alternativeembodiments, the functionality of one or more of the touch input module122, patient module 115, scan module 118, image processing module 125,and/or delivery module 120 may be combined into a single module ordivided into multiple additional modules.

User interface 112 may be a GUI that receives user commands and providesa graphical and/or audio output to a user. The user interface 112enables users to interact with intraoral scan application 108 throughmanipulation of graphical elements such as graphical icons and visualindicators such as buttons, menus, and so on. Intraoral scan application108 may include a number of modes, such as a planning mode, a scan mode,an image processing mode, and a delivery mode. The user interlace 112may display different graphical elements for each of the various modes.

Navigation or control of the user interface 122 of the intraoral scanapplication 108 may be performed via user input. The user input may beperformed through various devices, such as a touch input device (e.g., atouchscreen), keyboard, mouse, or other similar control devices. Use ofsuch devices may include the user sitting within arm's length reach ofthe computing device 105, which may be inconvenient when performingscanning. Alternatively, the user may also opt to physically move fromthe patient to the computing device 105 as necessary to navigate theuser interface or scan the patient, which may also be cumbersome.Navigation of the user interface may involve, for example, navigatingbetween various modules or modes, navigating between various segments,controlling the viewing of the 3D rendering, or any other user interfacenavigation. Such navigation can be an inefficient process due to theuser continuously disengaging and reengaging the patient. A touchsensitive scanner 150 allows the user to navigate or control the userinterface without continuously disengaging from the patient.

Touch input module 122 receives and interprets touch input data fromtouch sensitive scanner 150. Touch sensitive scanner 150 may receivedifferent types of touch input such as hold gestures, swipe gestures,tap gestures, circular gestures, and so on. Touch input module 122 maydetermine a type of touch gesture that a user performed based on thereceived touch input. Touch input module 122 may then initiate functionsor operations of the user interface (or intraoral scan applicationgenerally) responsive to the determined touch gesture. The functions oroperations that are initiated may depend both on the current mode of theintraoral scan application 108 and the determined touch gesture.Accordingly, the same touch gesture may cause a first function to beperformed in a first mode of the intraoral scan application and maycause a second function to be performed in a second mode. Specific modesof operation and touch gestures that initiate operations or functionsfor those modes are discussed in greater detail below.

Touch gestures may also be used to navigate between modes of operation.In one embodiment, the touch input module 122 enables a user to use thetouch sensitive scanner 150 to navigate through multiple levels ofcontrols using touch gestures. In one embodiment, a user uses up anddown swipe gestures to navigate through the levels of controls. Forexample, a user may provide an upward swipe gesture to navigate upwardone level and a downward swipe gesture to navigate down one level. Eachlevel of controls may provide a specific type of functionality, whichmay also depend on a current mode of operation of the intraoral scanapplication 108. Alternatively, upward and downward swipe gestures maybe used to navigate between modes of operation without the use ofmultiple levels of controls.

In one embodiment, an orientation of the touch sensitive scanner 150relative to a display of the computing device 105 is user configurable.Alternatively, an orientation of the touch sensitive scanner 150relative to the display may automatically be detected (e.g., by use of acamera or infrared sensor on the touch sensitive scanner 150 and/ordisplay). The different orientations may be a first orientation with aprobe of the touch sensitive scanner towards the display or a secondorientation with the probe directed away from the display. Depending onthe current orientation, one side of the touch sensor may be designatedas the left side and the other side may be designated as the right side.

In one embodiment, the multiple levels include at least a mode selectionlevel and a mode interaction level. While in the mode selection level, auser may provide left and right swipe gestures to navigate betweenmodes. From any mode of operation a user may provide an upward swipegesture to navigate from the mode interaction level for that mode to themode selection level. The user may then provide one or more sidewaysswipes to navigate to a new mode, and then provide a downward swipegesture to navigate to a mode interaction level for the current mode ofoperation. The available functions while in the mode interaction levelmay depend on the current mode of operation.

Patient module 115 provides a planning mode for intraoral scanapplication 108 that allows a user (e.g., dental practitioner) togenerate a patient profile and/or treatment plan for a patient. Thepatient profile may include information such as patient name, patientcontact information, patient dental history, and so on. The patient'sinformation may be entered into the intraoral scan application 108 bymeans of a keyboard or a touchscreen with a virtual on-screen keyboardon the user interface. The treatment plan may include dental proceduresto be performed and/or teeth to which the dental procedures are to beperformed. Some treatment plans include an indication of specificpatient teeth that are to be preparation teeth.

For many prosthodontic procedures (e.g., to create a crown, bridge,veneer, etc.), an existing tooth of a patient is ground down to a stump.The ground tooth is referred to herein as a preparation tooth, or simplya preparation. The preparation tooth has a finish line (also referred toas a margin line), which is a border between a natural (unground)portion of the preparation tooth and the prepared (ground) portion ofthe preparation tooth. The preparation tooth is typically created sothat a crown or other prosthesis can be mounted or seated on thepreparation tooth. In many instances, the finish line of the preparationtooth is below the gum line. While the term preparation typically refersto the stump of a preparation tooth, including the finish line andshoulder that remains of the tooth, the term preparation herein alsoincludes artificial stumps, pivots, cores and posts, or other devicesthat may be implanted in the intraoral cavity so as to receive a crownor other prosthesis. Embodiments described herein with reference to apreparation tooth also apply to other types of preparations, such as theaforementioned artificial stumps, pivots, and so on.

Once a patient profile and/or treatment plan are generated, intraoralscan application 108 may enter a scan mode provided by scan module 118.A user may transition from the planning mode to the scan mode byproviding touch input in the form of one or more swipe gestures. Thescan module 118 provides the scan mode, which allows the user to captureimages and/or video (e.g., for lower arch segment, upper arch segment,bite segment, and/or preparation tooth segments). The images and/orvideo may be used to generate a virtual 3D model of a dental site.

In one embodiment, the scan mode includes multiple scan segments, whichmay be dependent on the treatment plan. There may be a different scansegment for an upper arch (or portion thereof), a lower arch (or portionthereof), a bite, and/or one or more preparation teeth. In oneembodiment, a different scan segment is created for each preparationtooth. While in the mode interaction level and/or scan mode, the usermay provide touch gestures to navigate between scan segments that are tobe scanned (e.g., by providing left and right swipe gestures).

In one embodiment, touch input module 122 disables the touch sensor ofthe touch sensitive scanner 150 while a scan is being performed. Thetouch sensor may be disabled to ensure that the user does notinadvertently perform a touch gesture during a scan. In one embodiment,the touch sensor is automatically disabled when the touch sensitivescanner 150 detects an object in a field of view of a scanner head ofthe touch sensitive scanner 150. The scan mode is described in furtherdetail in FIG. 4 below.

Once scans for the various scan segments are complete, intraoral scanapplication 108 may enter an image processing mode provided by imageprocessing module 125. A user may transition from the scan mode to theimage processing mode by providing touch input in the form of one ormore swipe gestures. The image processing module 125 may process theintraoral scan data from the one or more scans of the various segmentsto generate a virtual 3D model of a scanned dental site.

In one embodiment, image processing module 125 performs imageregistration for each pair of adjacent or overlapping intraoral images(e.g., each successive frame of an intraoral video). Image registrationalgorithms are carried out to register two adjacent intraoral images,which essentially involves determination of the transformations whichalign one image with the other. Image registration may involveidentifying multiple points in each image (e.g., point clouds) of animage pair, surface fitting to the points of each image, and using localsearches around points to match points of the two adjacent images. Forexample, image processing module 125 may match points of one image withthe closest points interpolated on the surface of the other image, anditeratively minimize the distance between matched points. Imageprocessing module 125 may also find the best match of curvature featuresat points of one image with curvature features at points interpolated onthe surface of the other image, without iteration. Image processingmodule 125 may also find the best match of spin-image point features atpoints of one image with spin-image point features at pointsinterpolated on the surface of the other image, without iteration. Othertechniques that may be used for image registration include those basedon determining point-to-point correspondences using other features andminimization of point-to-surface distances, for example. Other imageregistration techniques may also be used.

Many image registration algorithms perform the fitting of a surface tothe points in adjacent images, which can be done in numerous ways.Parametric surfaces such as Bezier and B-Spline surfaces are mostcommon, although others may be used. A single surface patch may be fitto all points of an image, or alternatively, separate surface patchesmay be fit to any number of a subset of points of the image. Separatesurface patches may be fit to have common boundaries or they may be fitto overlap. Surfaces or surface patches may be fit to interpolatemultiple points by using a control-point net having the same number ofpoints as a grid of points being fit, or the surface may approximate thepoints by using a control-point net which has fewer number of controlpoints than the grid of points being fit. Various matching techniquesmay also be employed by the image registration algorithms.

In one embodiment, image processing module 125 may determine a pointmatch between images, which may take the form of a two dimensional (2D)curvature array. A local search for a matching point feature in acorresponding surface patch of an adjacent image is carried out bycomputing features at points sampled in a region surrounding theparametrically similar point. Once corresponding point sets aredetermined between surface patches of the two images, determination ofthe transformation between the two sets of corresponding points in twocoordinate frames can be solved. Essentially, an image registrationalgorithm may compute a transformation between two adjacent images thatwill minimize the distances between points on one surface, and theclosest points to them found in the interpolated region on the otherimage surface used as a reference.

Image processing module 125 may repeat image registration for alladjacent image pairs of a sequence of intraoral images to obtain atransformation between each pair of images, to register each image withthe previous one. Image processing module 125 then integrates all imagesinto a single virtual 30 model by applying the appropriate determinedtransformations to each of the images. Each transformation may includerotations about one to three axes and translations within one to threeplanes.

While in the image processing mode, a user may view the 3D model indetail to determine if it is acceptable. The image processing modeallows the dental practitioner to view the scans in detail at variousangles by rotating, moving, zooming in or out, etc. of the 3D model, Thedental practitioner may make a determination whether the quality of thescans are adequate, or whether particular segments or portions ofsegments should be rescanned. The dental practitioner may also navigateback to the scan mode to perform additional scans.

In one embodiment, the user may provide a hold gesture via the touchsensor. Responsive to the hold gesture, touch input module 122 mayactivate a gyroscope and/or an accelerometer of the touch sensitivescanner 150. While the gyroscope and/or accelerometer are active, touchinput module 122 may receive rotation and/or acceleration informationbased on a user moving the touch sensitive scanner 150. For example, theuser may reposition the touch sensitive scanner from a first orientationto a second orientation. Based on the rotation and/or accelerationinformation, intraoral scan application 108 may change a view of thevirtual 3D model from a first view having a first orientation of the 3Dmodel to a second view having a second orientation of the 3D model. Thechange from the first view to the second view may correspond to thechange from the first orientation to the second orientation of the touchsensitive scanner 150. In embodiments, a 3D rendering such as apreliminary or partial virtual 3D model may be created and updated asscan data is obtained. The above described hold gesture may also beperformed during the scan mode to change a view of the 3D rendering(e.g., the preliminary or partial virtual 3D model).

Once the scans are complete, the delivery module 120 provides a deliverymode that allows the user to send the scans and/or virtual 3D model outto an external facility to process the scans or 3D model. A user maytransition from the scan mode to the image processing mode by providingtouch input in the form of one or more swipe gestures.

The following non-limiting example may help understand the process morefully. A patient who wishes to straighten their teeth may opt forInvisalign® treatment. Invisalign is a process that creates a custommade series of clear aligners specifically for the patient. The clearaligners are worn over the patient's teeth and gradually shift thepatient's teeth. A new set of aligners may be worn after a specifiedperiod of time (e.g., two weeks) until treatment is complete. Thepatient may visit a dental practitioner or orthodontist to beginInvisalign treatment. The dental practitioner may utilize a scanningsystem, such as the iTero scanning system, to scan the patient's teethand generate 3D models used to create the clear aligners. The scanningsystem may be a system 100 which includes touch sensitive scanner 150coupled to a computing device 105 executing intraoral scan application108. The dental practitioner would begin the Invisalign treatment byentering the patient's information into a patient profile and/orcreating a treatment plan in the planning mode. The Invisalign treatmentmay call for a scan of the patient's lower arch, upper arch, and bitesegments. Once the dental practitioner completes the patient profileand/or treatment plan, the dental practitioner may navigate to the scanmode to begin scanning. The scan mode may present a user interface tothe dental practitioner similar to user interface 400 of FIG. 4 to bediscussed in further detail below.

The dental practitioner may use touch sensitive scanner 150 (e.g.,intraoral scanner) to capture the patient's teeth segments (e.g., upperarch, lower arch, bite segments) in one or more sets of intraoralimages. The scan module 118 may register and stitch together theintraoral images to create a 3D rendering of the scanned segments andpresent the 3D renderings to the dental practitioner on the userinterface of the intraoral scan application 108. Once the scans arecompleted, the dental practitioner may next navigate to the imageprocessing mode, which may generate a virtual 3D model by registeringand stitching together the intraoral images. Once an adequate set of 3Drenderings and/or virtual 3D model are complete, the 3D renderings maybe saved to the patient profile. The dental practitioner may thennavigate to the delivery mode to electronically send the completedpatient profile to a processing center. The processing center may thengenerate the custom made series of clear aligners for the patient anddeliver the clear aligners to the dental practitioner. The patient wouldthen return to the dental practitioner to receive the first set of clearaligners and verify the clear aligners properly fit onto the patient'steeth.

FIG. 2 illustrates a perspective view of an intraoral scanner 200 withtouch sensitive input. The intraoral scanner 200 may alternatively be amedical scanning device for scanning objects other than an intraoralcavity. Other types of medical scanning devices 200 to which embodimentsof the present invention may apply include other types of opticalscanners, x-ray devices, ultrasound devices, and so on. Each suchmedical scanning device may include at the least an image sensor togenerate medical images, a communication module to transmit the medicalimages to a computing device, and a touch sensor usable to manipulatethe medical images on the computing device and/or a representation of ascanned object generated from the medical images. These components maybe coupled together directly or via a bus. The touch sensor may also beusable to navigate a user interface of a medical scan applicationrunning on the computing device. The medical scanning devices mayadditionally include one or more buttons that may be used both toinitiate generation of the medical images and to activate and/ordeactivate the touch sensor.

In one embodiment, intraoral scanner 200 corresponds to touch sensitivescanner 150 of FIG. 1. The intraoral scanner 200 may include a probe 210that protrudes from one end of a body of the intraoral scanner 200. Theprobe 210 may include a scanner head 220 that captures optical data andprovides the optical data to one or more optical sensors disposed withinthe intraoral scanner 200.

In one embodiment, intraoral scanner 200 includes a semiconductor laserunit that emits a focused light beam. The light beam may pass through anillumination module disposed within the intraoral scanner 200, whichsplits the light beam into an array of incident light beams. Theillumination module may be, for example, a grating or a micro lens arraythat splits the light beam into an array of light beams. In oneembodiment, the array of light beams is an array of telecentric lightbeams. Alternatively, the array of light beams may not be telecentric.

Intraoral scanner 200 may further include a unidirectional mirror orbeam splitter (e.g., a polarizing beam splitter) that passes the arrayof light beams. A unidirectional mirror allows transfer of light fromthe semiconductor laser through to downstream optics, but reflects lighttravelling in the opposite direction. A polarizing beam splitter allowstransfer of light beams having a particular polarization and reflectslight beams having a different (e.g., opposite) polarization. In oneembodiment, as a result of a structure of the unidirectional mirror orbeam splitter, the array of light beams will yield a light annulus on anilluminated area of an imaged object within a field of view of thescanner head 220 as long as the area is not in focus. Moreover, theannulus will become a completely illuminated spot once in focus. Thisensures that a difference between measured intensities of out-of focuspoints and in-focus points will be larger.

Along an optical path of the array of light beams after theunidirectional mirror or beam splitter, intraoral scanner 200 mayinclude confocal focusing optics, and probe 210 (also referred to as anendoscopic probing member). Additionally, a quarter wave plate may bedisposed along the optical path after the unidirectional mirror or beamsplitter to introduce a certain polarization to the array of lightbeams. In some embodiments this may ensure that reflected light beamswill not be passed through the unidirectional mirror or beam splitter.

The probe 210 may internally include a rigid, light-transmitting medium,which may be a hollow object defining within it a light transmissionpath or an object made of a light transmitting material, e.g. a glassbody or tube. In one embodiment, the probe 210 includes a prism such asa folding prism. At the end of the probe 210 where the scanner head 220is located, the probe 210 may include a mirror of the kind ensuring atotal internal reflection. Thus, the mirror may direct the array oflight beams towards a teeth segment or other object. The scanner head220 thus emits array of light beams, which impinge on to surfaces ofscanned objects such as teeth.

The array of light beams may be are arranged in an X-Y plane. As thesurface on which the incident light beams hits is an uneven surface,illuminated spots are displaced from one another along the Z axis, atdifferent (X_(i), Y_(i)) locations. Thus, while a spot at one locationmay be in focus of the confocal focusing optics, spots at otherlocations may be out-of-locus. Therefore, the light intensity ofreturned light beams of the focused spots will be at its peak, while thelight intensity at other spots will be off peak. Thus, for eachilluminated spot, multiple measurements of light intensity are made atdifferent positions along the Z-axis. For each of such (X_(i), Y_(i))location, the derivative of the intensity over distance (Z) may be made,with the Z_(i) yielding maximum derivative, Z₀, being the in-focusdistance. As pointed out above, the incident light from the array oflight beams forms a light disk on the surface when out of focus and acomplete light spot when in focus. Thus, the distance derivative will belarger when approaching in-focus position, increasing accuracy of themeasurement.

The light scattered from each of the light spots includes a beamtravelling initially in the Z axis along the opposite direction of theoptical path traveled by the array of light beams. Each returned lightbeam in an array of returning light beams corresponds to one of theincident light beams in array of light beams. Given the asymmetricalproperties of unidirectional mirror or beam splitter, the returned lightbeams are reflected in the direction of detection optics (e.g., one ormore optical sensors).

The optical sensor may be an image sensor having a matrix of sensingelements each representing a pixel of the image. In one embodiment, theoptical sensor is a charge coupled device (CCD) sensor. In oneembodiment, the optical sensor is a complementary metal-oxidesemiconductor (CMOS) type image sensor. Other types of image sensors mayalso be used. The optical sensor detects light intensity at each pixel.

Intraoral scanner 200 may further include a control module connectedboth to the semiconductor laser and a motor, voice coil or othertranslation mechanism. The motor may be linked to confocal focusingoptics for changing a focusing setting of confocal focusing optics.After receipt of feedback that the location of the one or more lenseshas changed, the control module may induce the laser to generate a lightpulse. The control unit may additionally synchronize the image-capturingmodule to receive and/or store data representative of the lightintensity from each of the sensing elements at the particular locationof the one or more lenses (and thus of the local depth). In subsequentsequences, the location of the one or more lenses (and thus the focaldepth) will change in the same manner and the data capturing willcontinue over a wide focal range of confocal focusing optics.

One embodiment of an intraoral scanner 200 that uses confocal imaging togenerate 3D images has been described above. However, embodiments shouldbe understood as covering all types of 3D imaging devices. For example,intraoral scanner 200 may include stereoscopic camera pairs forgenerating stereoscopic images, may use projections of structured lightpatterns to determine 3D information, and so on.

As illustrated, intraoral scanner 200 additionally includes multipleinput devices. These input devices may include one or more buttons 240and a touch sensor 230. In one embodiment, intraoral scanner 200includes a pair of buttons disposed at opposite sides of the intraoralscanner 200. One of the buttons is hidden in the view of intraoralscanner 200 shown in FIG. 2. In one embodiment, each button 240 of thepair of buttons generates the same signals, and thus causes the sameoperations or functions to be performed. Depending on how a user holdsintraoral scanner 200, one of the buttons may be more convenient topress than the other button. In one embodiment, simultaneously pressingthe pair of buttons activates touch sensor 230. Alternatively, otherbutton push combinations of one or both of the buttons may activate thetouch sensor 230.

Touch sensor 230 is capable of detecting a user touch, and can receivemultiple different types of touch gestures, such as swipes, holds, taps,and so on. A hold gesture may be detected when a user presses the touchsensor 230 for a threshold amount of time (e.g., a press duration of 1second, 2 seconds, etc.). A swipe gesture may be detected when a userinitially presses one side of the touch sensor 230 with a finger, andthen moves their finger across to another side of the touch sensor 230.Touch sensor 230 may be a touchpad, trackpad or touch screen that cantranslate the motion and position of a user's fingers into touchgestures. Touch sensor 230 may include capacitive sensing elements,conductance sensing elements, infrared sensing elements, inductivesensing elements, and/or other sensing elements. Touch sensor 230 may bea one dimensional touchpad or a two dimensional touchpad.

The touch sensor 230 may detect a touch input gesture including, but notlimited to, swipe gestures, hold gesture, single-finger touch gestures,and/or multi-finger touch gestures. Responsive to detecting a touchinput, touch sensor 230 may generate touch input data (also referred toas a touch input signal) for a particular touch gesture and transmit thetouch input data to a connected computing device. The touch inputgesture may be used to control the user interface of the intraoral scanapplication 108. For example in scan mode, a swipe left and swipe rightgesture may allow the user to navigate between teeth segments (e.g.,lower arch segment, upper arch segment, and bite segment). A swipe upand swipe down gesture may allow the user to navigate between thevarious modules of the intraoral scan application 108 (e.g., patientmodule, scan module, image processing module, delivery module). Thetouch input gesture may also be used to manipulate medical data that hasbeen generated by the intraoral scanner 200 and sent to a computingdevice. For example, a hold gesture may allow the user to activate aninertial measurement device and rotate a 3D rendering generated frommedical images by rotating the intraoral scanner 200. The touch sensor230 and at least one button 240 may be used in conjunction to performadditional control of the user interface, medical images, and/orrepresentations generated from the medical images. For example, uponholding the button 240 in conjunction with a swipe up gesture, swipedown gesture or side swipe gesture on the touch sensor 230, theintraoral scan application 108 may launch an overlay mode similar tothat shown in FIG. 7. In another example, holding the button 240 inconjunction with a swipe up gesture, swipe down gesture or side swipegesture may cause the 3D rendering to zoom in or out.

In one embodiment, intraoral scanner 200 disables touch sensor 230during scanning to ensure that a user will not inadvertently issuecommands via the touch sensor 230. Accordingly, intraoral scanner 200may automatically disable touch sensor 230 when an object is detected ina field of view of sensor head 220 and/or within a threshold distancefrom sensor head 230. Intraoral scanner 200 may additionally include oneor more lights (e.g., light emitting diodes (LEDs)), which may bemounted to probe 210. These lights may be automatically activated whenan object is detected in the field of view of the sensor head 220.

In one embodiment, intraoral scanner 200 includes an inertialmeasurement device. The inertial measurement device may include one ormore accelerometers and/or one or more gyroscopes, that may helpdetermine the intraoral scanner's 200 velocity, orientation, rotation,movement, and/or gravitational forces. Additionally, an inertialmeasurement device may include a magnetic sensor. A magnetic sensor mayallow the inertial measurement device to determine the rotation positionof the intraoral scanner 200. Raw inertial measurement data may beprocessed to help determine the orientation of intraoral scanner 200.Raw inertial measurement data and processed inertial measurement datamay be referred to as inertial measurement data.

In one embodiment, intraoral scanner 200 activates the inertialmeasurement device (or devices) responsive to touch sensor 230 detectinga hold gesture. In one embodiment, while the user continues the holdgesture, inertial measurement data generated by the inertial measurementdevice is used to control the view and orientation of a virtual 3Dmodel. Alternatively, once the hold gesture is received, the inertialmeasurement data may continue to be used for controlling the view andorientation of the virtual 3D model.

FIG. 3A illustrates a flow diagram for a method 300 of receiving a touchinput from a touch sensitive intraoral scanner and performing adetermined function to control a user interface of an intraoral scanapplication during an intraoral scan session, in accordance withembodiments of the present invention. This method may be performed byprocessing logic that comprises hardware (e.g., circuitry, dedicatedlogic, programmable logic, microcode, etc.), software (such asinstructions run on a processing device), or a combination thereof. Inone embodiment, processing logic corresponds to computing device 105 ofFIG. 1 (e.g., to a computing device 105 executing intraoral scanapplication 108).

By way of non-limited example, method 300 may occur during an oral scansession of a patient by a dental practitioner. At block 305, processinglogic receives a touch input from a touch sensitive intraoral scannerduring a medical scan session 305. The touch input may include a swipegesture, a hold gesture, a tap gesture, or some other gesture. At block310, processing logic determines whether the touch input gesture is oneof a swipe gesture or a hold gesture.

At block 312, upon determining the touch input gesture is a swipegesture, processing logic determines a swipe direction of the swipegesture. The swipe direction may include, but is not limited to, a swipeleft gesture, a swipe right gesture, a swipe up gesture, and a swipedown gesture. The swipe direction may also be a diagonal swipedirection.

At block 314, processing logic determines a current mode of theintraoral scan application. For example, the current mode may be aplanning mode, a scanning mode, an image processing mode or a deliverymode. Determining the current mode of the intraoral scan application mayinclude determining an active module and a current focus of the activemodule. In one embodiment, the active module may be patient module, scanmodule, image processing module, or delivery module. The current focusof the active module describes an active or highlighted section of theactive module. For example, a dental practitioner preparing to scan thelower arch segment of a patient may navigate to the lower arch segmentof a scan mode presented by the scan module. In such a case, the lowerarch segment may be the current focus of the active module, and the scanmodule may be the active module. For example, FIG. 4, which will bedescribed in further detail below, provides an illustration of thecurrent focus of the active module (e.g., lower arch segment) and theactive module (e.g., scan mode) denoted by highlights around theaforementioned sections.

At block 316, processing logic determines a function to perform based onswipe direction and current mode. The function to perform may include anaction or operation which may cause the current focus of the activemodule to move to a next focus. For example, if the current focus is theupper arch segment of the scan mode, processing logic may determine thata swipe right gesture will change the current focus to the next segment(e.g., bite segment) in scan mode, or a swipe left gesture may changethe current focus to the previous segment (e.g., lower arch segment) inscan mode. At block 318, processing logic performs the determinedfunction (e.g., switches to the next or previous focus).

The function that is determined at block 316 may also depend on acurrent active level. In one embodiment, processing logic providesmultiple levels that may include, for example, a mode selection leveland a mode interaction level. While in the mode selection level, a usermay provide left and right swipe gestures to navigate between modes.While in a mode interaction level, a user may provide left and rightswipe gestures to switch a focus to a next or previous segment, forexample.

At block 320, upon determining the touch input gesture is a holdgesture, processing logic activates an inertial measurement device inthe intraoral scanner. Once activated, the inertial measurement devicemay generate inertial measurement data (e.g., velocity, orientation,gravitational forces, and/or rotational position) of the intraoralscanner. At block 322, processing logic receives inertial measurementdata from the intraoral scanner that indicates a change of the intraoralscanner from a first orientation to a second orientation. For example,processing logic may receive data from the intraoral scanner that theintraoral scanner was rotated 45 degrees in a downward direction. Atblock 324, processing logic may perform a view function to change from afirst view of a 3D model to a second view of the 3D model, wherein thechange from the first view to the second view corresponds to the changefrom the first orientation to the second orientation of the intraoralscanner. For example, upon receiving inertial measurement data that theintraoral scanner was rotated 45 degrees in a downward direction,processing logic may rotate the 3D rendering of an upper arch segment 45degrees in a downward direction in correspondence with the inertialmeasurement data.

FIG. 3B illustrates a flow diagram for a method 350 of enabling anddisabling the touch sensor of a touch sensitive intraoral scanner andperforming functions based on inputs from the touch sensor, inaccordance with embodiments of the present invention. This method may beperformed by processing logic that comprises hardware (e.g., circuitry,dedicated logic, programmable logic, microcode, etc.), software (such asinstructions run on a processing device), or a combination thereof. Inone embodiment, processing logic corresponds to intraoral scanner 200 ofFIG. 2.

A touch sensor of the intraoral scanner may be disabled to prevent theuser from performing unintended touch input. For example, a dentalpractitioner handling the intraoral scanner may accidentally brush afinger against the touch sensor during the course of a scan, which couldcause the intraoral scan application to perform a determined function ifthe touch sensor is active.

At block 355, processing logic may detect user input to enable the touchsensor. In one embodiment, the intraoral scanner may include twobuttons. For example, the user input to enable the touch sensor may comein the form of the simultaneous pressing of the two buttons. If no userinput is detected to enable the touch sensor, processing logic moves toblock 390. If the user input to enable the touch sensor is detected, thetouch sensor is enabled at block 360.

At block 365, processing logic detects whether an object is in a fieldof vision (FOV) of scanner head of the intraoral scanner. For example, apatient's teeth may be detected when an intraoral scanner is insertedinto the patient's mouth. In one embodiment, the scanner head's FOV is avariable distance. For example, the FOV may be configured as 13.5millimeters×13.5 millimeters×13.5 millimeters (13.5 cubic millimeters),but may also be increased or decreased as needed. An ideal FOV formedical scans may be dependent on the type of scan. For intraoral scans,a FOV of less than 20 cubic millimeters may be appropriate. Processinglogic may also determine at block 365 whether a detected object iswithin a threshold distance from the scanner head.

If no object is detected in the FOV of the scanner head, processinglogic moves to block 370, where processing logic detects a gesture viathe touch sensor. At block 385, if a swipe gesture (e.g., swipe leftgesture, swipe right gesture, swipe up gesture, swipe down gesture) isdetected, processing logic sends the swipe gesture to the computingdevice; processing logic then returns to block 365. If a hold gesture isdetected, processing logic activates the inertial measurement device atblock 380. At block 392, processing logic generates inertial measurementdata and sends the data to the computing device; processing logic thenreturns to block 365.

Returning to block 365, if an object is detected in the FOV of thescanner head, processing logic moves to block 375 and disables the touchsensor and activates light(s) such as light emitting diodes (LEDs) thatmay be mounted to a probe of the intraoral scanner. For example, oncethe user inserts the intraoral scanner into the patient's mouth forscanning, the touch sensor may be disabled to prevent unintended touchinput and lights near the scanner head may illuminate to providelighting to the scanning area.

At block 390, processing logic detects user input to begin scanning. Insome embodiments, the user input may be a press of a button of theintraoral scanner. If no user input to begin scanning is detected,processing logic returns to block 355. At block 393, upon detection ofuser input to begin scanning, processing logic generates intraoralimages and/or video. For example, a dental practitioner may press abutton of the intraoral scanner to begin an intraoral scan of a bitesegment, for which intraoral images and/or video may be generated. Themethod may then return to block 355. Method 350 may continue until auser turns off the intraoral scanner.

FIG. 3C illustrates a flow diagram for a method 394 of generatingmedical images, providing the medical images to a computing device, andthen manipulating those medical images on the computing device, inaccordance with embodiments of the present invention. This method may beperformed by a medical scanning device. In one embodiment, the medicalscanning device corresponds to intraoral scanner 200 of FIG. 2.Alternatively, the medical scanning device may be another type ofmedical imaging device.

At block 395 of method 394, an image sensor of the medical scanningdevice generates one or more medical images. The image sensor may be anx-ray device, an ultrasound transceiver, an optical image sensor (e.g.,CCD sensor, a CMOS type image sensor, etc.), or other type of imagesensor.

At block 396, a communication module of the medical scanning devicetransmits the medical images to a computing device that is connected tothe medical scanning device via a wired or wireless connection. Thecommunication module may be a wireless communication module (e.g., aWi-Fi network adapter, a Bluetooth transceiver, a Zigbee transceiver, aninfrared transceiver, an ultrasound transceiver, etc.) or a wiredcommunication module (e.g., an Ethernet network adapter, a USB module,etc.).

At block 397, the medical scanning device activates a touch sensor ofthe medical scanning device. The touch sensor may have been disabledwhile the medical images were being generated so as not to interferewith the image generation process (e.g., from an unintentional touchgesture). The touch sensor may be activated responsive to a userpressing one or more buttons of the medical scanning device. In oneembodiment, a particular button push combination of a pair of buttonsactivates the touch sensor. For example, a simultaneous button push ofthe pair of buttons may activate the touch sensor.

At block 398, the medical scanning device receives a touch input (e.g.,detects a user finger on the touch sensor). At block 399, the touchsensor may determine a touch gesture represented by the touch input, andmay generate a corresponding touch input data or signal. The medicalscanning device may then send the touch input data to the connectedcomputing device. The touch input data may cause a medical scanapplication running on the computing device to perform one or morefunctions that may manipulate the medical images, manipulate arepresentation (e.g., virtual 3D model) generated from the medicalimages, navigate a UI of the medical scan application, and so on.

FIG. 4 illustrates an example scan mode user interface (UI) 400 of anintraoral scan application (e.g., of intraoral scan application 108 ofFIG. 1). The UI 400 may be interactive, and user interactive options maybe represented as icons. The user may interact with the UI 400 byvarious input (e.g., mouse, keyboard, touchscreen, touch sensors from anintraoral scanner, or other similar devices).

At the top of the UI 400, icons for the patient information mode 410,scan mode 415, image processing mode 420, and delivery mode 425 aredisplayed. The active mode (e.g., scan mode 415 in this instance) ishighlighted. The user may switch to a different mode by clicking thedesired icon (e.g., with a mouse) or touching the icon (e.g., with atouchscreen). The user may alternatively utilize the touch input of theintraoral scanner to switch to a different mode. For example, a swipe upgesture may change from a mode interaction level to a mode selectionlevel. Once in the mode selection level, a left or right swipe gesturemay navigate between modes.

Various teeth segments are displayed on the upper left section of the UI400. The upper arch segment 430, bite segment 435, lower arch segment440, and an individual preparation tooth segment 445 are represented byicons. Depending on a treatment plan for a current patient, there may beno preparation tooth segments, one preparation tooth segment, ormultiple preparation tooth segments. The preparation tooth segment 445may be represented by any individual tooth corresponding to a patient'stooth that will be used as a preparation tooth. Alternatively, an upwardswipe gesture may transition the intraoral scan application to a nextmode.

While an intraoral scan application is in the mode interaction level fora scan mode, a user may generate intraoral data sets for each of thesegments 430, 435, 440, 445. A user may select which of the segments togenerate an intraoral data set for using any number of input devices.One such input device is the touch sensor integrated into the intraoralscanner. The segment that is the current focus is highlighted with anoutline (e.g., lower arch segment 440). A segment description 460provides a textual description of the segment that has the current focus(e.g., scan lower arch). The previous segment button 465 and nextsegment button 455 are user interactive, and allow the user to move thecurrent focus to another tooth segment. A user may navigate betweensegments by using a touch screen or mouse to press the next segmentbutton 455 or the previous segment button 465. Alternatively, the usermay use the touch sensor of the intraoral scanner to input a swipegesture. The focus may then switch to the next or previous segment basedon the swipe gesture. For example, where the current focus is the upperarch segment 430, a swipe right gesture will change the current focus tothe bite segment 435, or a swipe left gesture will change the currentfocus to the lower arch segment 440. FIG. 6 provides a furtherillustration of utilizing swipe gestures to change the current focus.

A viewing window 470 provides a live view of the objects in the field ofview of the intraoral scanner. The viewing window 470 may provide anindication of images and/or video to be captured.

A 3D rendering 480 provides a rough virtual 3D model generated bystitching together images from an image data set generated for thesegment that has the current focus (e.g., lower arch segment 440). Auser may view the 3D model from various angles, and may zoom in or out.The user may use a touchscreen on the computing device 105 to move,rotate, and zoom in/out of the 3D model. The user may also utilize thetouch sensor of the intraoral scanner to move, rotate and zoom in/out ofthe 3D model by activating and using data collected by the inertialmeasurement device as described in blocks 320-324 of FIG. 3A, andfurther described in FIG. 5.

FIG. 5 illustrates a view function that triggers an example rotation ofan intraoral scanner 520 and the resulting rotation of a virtual 3Dmodel 530 generated based on an intraoral scan. A hold gesture 510 maybe detected by a touch sensor of an intraoral scanner 200, which mayactivate an inertial measurement device of the intraoral scanner 200.Once the inertial measurement device is activated, inertial measurementdata (e.g., rotational and/or acceleration data) may be generated as auser moves (e.g., rotates) the intraoral scanner 200. This inertialmeasurement data may be used to rotate and/or move the virtual 3D model.The virtual 3D model may then move concurrently with the intraoralscanner. For example, as the intraoral scanner 200 moves from a firstorientation 522 to a second orientation 524 and on to a thirdorientation 526, the 3D model will move from a first view 535corresponding to the first orientation 522 to a second view 540corresponding to the second orientation 524, and on to a third view 545corresponding to the third orientation 526. The intraoral scanner 200may be rotated, moved in various directions, etc., and the 3D model willalso move, rotate, etc. accordingly. Therefore, the dental practitionermay conveniently control the 3D model while still holding the intraoralscanner 200 without having to physically move within proximity of acomputing device to utilize a mouse or touchscreen.

FIG. 6 illustrates example swipe gestures and resulting functionsperformed by an intraoral scan application in a scan mode of operation.A focus of the scan mode may initially be a lower arch segment 634, asshown in first view 630. The lower arch segment 634 is highlighted and alower arch 3D rendering 632 is displayed in the first view 630.

The user may perform a swipe right gesture 610 on the intraoral scanner200, which causes the Intraoral scan application 108 to move the currentfocus to the upper arch segment 644, as shown in second view 640. In thesecond view 640, the upper arch segment 644 is highlighted and the upperarch 3D rendering 642 is displayed.

The user may perform a swipe left gesture 615 to return the focus to thelower arch segment 630. Alternatively, the user may perform a swiperight gesture 610 to move the focus to the bite segment 650, as shown inthird view 650. In the third view 650, the bite segment 654 ishighlighted and a bite 3D rendering 652 is displayed.

FIG. 7 illustrates an example overlay 700 for an intraoral scanapplication 108. The overlay 700 may appear over various active modes(e.g., scan mode) and may provide assistance information regarding touchgestures that may be used for the particular active mode. For example,in the illustration overlay 700 provides instructional informationregarding the use of the intraoral scanner 200 and available controlfunctionality. Swipe gesture instructions 705 may provide the userinformation on a command that will be entered responsive to a swipe leftgesture (e.g., a “Prev” or previous command) and a swipe right gesture(e.g., a “Next” command). Hold gesture instructions 710 may provide theuser information on a command that will be entered responsive to a holdgesture (e.g., “Press and hold to rotate the model”). Overlay 700 mayalso provide additional information to assist the user in the scanningprocess, such as guidance as to scanning technique, and/or may highlightscan assistance indications (e.g., ones corresponding to missing and/orflawed scan data).

FIG. 8 illustrates a diagrammatic representation of a machine in theexample form of a computing device 800 within which a set ofinstructions, for causing the machine to perform any one or more of themethodologies discussed herein, may be executed. In alternativeembodiments, the machine may be connected (e.g., networked) to othermachines in a Local Area Network (LAN), an intranet, an extranet, or theInternet. The machine may operate in the capacity of a server or aclient machine in a client-server network environment, or as a peermachine in a peer-to-peer (or distributed) network environment. Themachine may be a personal computer (PC), a tablet computer, a set-topbox (STB), a Personal Digital Assistant (PDA), a cellular telephone, aweb appliance, a server, a network router, switch or bridge, or anymachine capable of executing a set of instructions (sequential orotherwise) that specify actions to be taken by that machine. Further,while only a single machine is illustrated, the term “machine” shallalso be taken to include any collection of machines (e.g., computers)that individually or jointly execute a set (or multiple sets) ofinstructions to perform any one or more of the methodologies discussedherein.

The example computing device 800 includes a processing device 802, amain memory 804 (e.g., read-only memory (ROM), flash memory, dynamicrandom access memory (DRAM) such as synchronous DRAM (SDRAM), etc.), astatic memory 806 (e.g., flash memory, static random access memory(SRAM), etc.), and a secondary memory (e.g., a data storage device 828),which communicate with each other via a bus 808.

Processing device 802 represents one or more general-purpose processorssuch as a microprocessor, central processing unit, or the like. Moreparticularly, the processing device 802 may be a complex instruction setcomputing (CISC) microprocessor, reduced instruction set computing(RISC) microprocessor, very long instruction word (VLIW) microprocessor,processor implementing other instruction sets, or processorsimplementing a combination of instruction sets. Processing device 802may also be one or more special-purpose processing devices such as anapplication specific integrated circuit (ASIC), a field programmablegate array (FPGA), a digital signal processor (DSP), network processor,or the like. Processing device 802 is configured to execute theprocessing logic (instructions 826) for performing operations and stepsdiscussed herein.

The computing device 800 may further include a network interface device822 for communicating with a network 864. The computing device 800 alsomay include a video display unit 810 (e.g., a liquid crystal display(LCD) or a cathode ray tube (CRT)), an alphanumeric input device 812(e.g., a keyboard), a cursor control device 814 (e.g., a mouse), a touchinput device 816 (e.g., a touchscreen), and a signal generation device820 (e.g., a speaker).

The data storage device 828 may include a machine-readable storagemedium (or more specifically a non-transitory computer-readable storagemedium) 824 on which is stored one or more sets of instructions 826embodying any one or more of the methodologies or functions describedherein. A non-transitory storage medium refers to a storage medium otherthan a carrier wave. The instructions 826 may also reside, completely orat least partially, within the main memory 804 and/or within theprocessing device 802 during execution thereof by the computer device800, the main memory 804 and the processing device 802 also constitutingcomputer-readable storage media.

The computer-readable storage medium 824 may also be used to store anintraoral scan application 850, which may correspond to the similarlynamed component of FIG. 1. The computer readable storage medium 824 mayalso store a software library containing methods for an intraoral scanapplication 850. While the computer-readable storage medium 824 is shownin an example embodiment to be a single medium, the term“computer-readable storage medium” should be taken to include a singlemedium or multiple media (e.g., a centralized or distributed database,and/or associated caches and servers) that store the one or more sets ofinstructions. The term “computer-readable storage medium” shall also betaken to include any medium other than a carrier wave that is capable ofstoring or encoding a set of instructions for execution by the machineand that cause the machine to perform any one or more of themethodologies of the present invention. The term “computer-readablestorage medium” shall accordingly be taken to include, but not belimited to, solid-state memories, and optical and magnetic media.

It is to be understood that the above description is intended to beillustrative, and not restrictive. Many other embodiments will beapparent upon reading and understanding the above description. Althoughembodiments of the present invention have been described with referenceto specific example embodiments, it will be recognized that theinvention is not limited to the embodiments described, but can bepracticed with modification and alteration within the spirit and scopeof the appended claims. Accordingly, the specification and drawings areto be regarded in an illustrative sense rather than a restrictive sense.The scope of the invention should, therefore, be determined withreference to the appended claims, along with the full scope ofequivalents to which such claims are entitled.

What is claimed is:
 1. A method for scanning multiple intraoral segmentsof a patient's oral cavity, the method comprising: displaying to adisplay a first indication of a first segment of the patient's oralcavity to be scanned; receiving a first user input via a hand heldintraoral scanning device; responsive to receiving the first user input,causing the hand held intraoral scanning device to enter a scan mode;responsive to the hand held intraoral scanning device scanning the firstsegment of the patient's oral cavity, receiving one or more intraoralscans of the first segment; generating a three-dimensional (3D) model ofthe first segment based on the one or more intraoral scans; displayingto the display the 3D model of the first segment; receiving a seconduser input via the intraoral scanning device after receiving the one ormore intraoral scans of the first segment; responsive to receiving thesecond user input, displaying to the display an overlay including afirst element corresponding to third user input for navigating to a nextsegment and a second element corresponding to fourth user input fornavigating to a previous segment; receiving, after displaying theoverlay, the third user input for navigating to the next segment via thehand held intraoral scanning device; displaying to the display a secondindication of a second segment of the patient's oral cavity to bescanned; entering the scan mode after receiving the third user input fornavigating to the next segment; responsive to the hand held intraoralscanning device scanning the second segment of the patient's oralcavity, receiving one or more intraoral scans of the second segment;generating a 3D model of the second segment based on the one or moreintraoral scans of the second segment; and displaying to the display the3D model of the second segment.
 2. The method of claim 1, wherein theoverlay includes a third element corresponding to fifth user input forentering a mode for rotating the displayed 3D model of the first segmentusing data from an inertial measurement device of the hand heldintraoral scanning device, and wherein the method further comprises:receiving, via the hand held intraoral scanning device, the fifth userinput for entering the mode for rotating the displayed 3D model of thefirst segment using the data from the inertial measurement device of thehand held intraoral scanning device; detecting a movement of theintraoral scanning device via the inertial measurement device; andresponsive to receiving the fifth user input for entering the mode forrotating the displayed 3D model of the first segment using the data fromthe inertial measurement device of the hand held intraoral scanningdevice and detecting the movement, performing the following comprising:updating an orientation of the 3D model of the first segment based onthe detected movement; and displaying the 3D model of the first segmentat the updated orientation.
 3. The method of claim 2, wherein the fifthuser input for entering the mode for rotating the displayed 3D model ofthe first segment using the data from the inertial measurement device ofthe hand held intraoral scanning device is a hold gesture.
 4. The methodof claim 1, wherein the first user input and the second user input aretouch gestures.
 5. The method of claim 1, further comprising:determining that the first user input comprises a first gesture; anddetermining that the second user input comprises a second gesture. 6.The method of claim 1 further comprising: receiving, after displayingthe overlay, the fourth user input for navigating to the previoussegment via the hand held intraoral scanning device; entering the scanmode after receiving the fourth user input for navigating to theprevious segment; and responsive to the hand held intraoral scanningdevice rescanning the first segment of the patient's oral cavity,receiving one or more additional intraoral scans of the first segment;and revising the 3D model of the first segment based on the one or moreadditional intraoral scans of the first segment.
 7. The method of claim1, wherein the displaying of the overlay responsive to the receiving thesecond user input comprises displaying the overlay on the displayed 3Dmodel of the first segment.
 8. A system for scanning multiple intraoralsegments of a patient's oral cavity, the system comprising: a hand heldintraoral scanning device comprising: a body; one or more buttonsdisposed on the body, wherein the one or more buttons are to receive afirst user input, a second user input, and a third user input; and aprobe at a first end of the body, the probe comprising a scanner head;wherein the hand held intraoral scanning device is to: receive the firstuser input via the one or more buttons; enter a scan mode to beginscanning a first segment of the patient's oral cavity responsive toreceiving the first user input; generate one or more intraoral scans ofthe first segment; transmit the one or more intraoral scans of the firstsegment; receive the second user input via the one or more buttons aftertransmitting the one or more intraoral scans of the first segment; sendan indication of the second user input; receive the third user input viathe one or more buttons after sending the indication of the second userinput; enter the scan mode to begin scanning a second segment of thepatient's oral cavity responsive to receiving the third user input;generate one or more intraoral scans of the second segment; and transmitthe one or more intraoral scans of the second segment; and anon-transitory computer-readable storage medium having instructionsthat, when executed by a processor, cause the processor to: display to adisplay a first indication of the first segment of the patient's oralcavity to be scanned; receive the one or more intraoral scans of thefirst segment from the hand held intraoral scanning device; generate athree-dimensional (3D) model of the first segment based on the one ormore intraoral scans; display to the display the 3D model of the firstsegment; receive the second user input from the hand held intraoralscanning device; responsive to receiving the second user input, displayto the display an overlay including a first element corresponding tothird user input for navigating to a next segment and a second elementcorresponding to fourth user input for navigating to a previous segment,wherein the third user input for navigating to the next segment isreceived after displaying the overlay; display to the display a secondindication of a second segment of the patient's oral cavity to bescanned; receive the one or more intraoral scans of the second segment;generate a 3D model of the second segment based on the one or moreintraoral scans of the second segment; and display to the display the 3Dmodel of the second segment.
 9. The system of claim 8, wherein the handheld intraoral scanning device further comprises an inertial measurementdevice disposed within the body, wherein: the overlay includes a thirdelement corresponding to fifth user input for entering a mode forrotating the displayed 3D model of the first segment using data from theinertial measurement device; the one or more buttons are to receive thefifth user input for entering the mode for rotating the displayed 3Dmodel of the first segment using the data from the inertial measurementdevice of the hand held intraoral scanning device; the inertialmeasurement device is to detect movement of the hand held intraoralscanning device and is to generate inertial measurement data indicatingthe movement; and responsive to the processor receiving the fifth userinput for entering the mode for rotating the displayed 3D model of thefirst segment and the inertial measurement data, the instructions causethe processor to update an orientation of the 3D model of the firstsegment based on the detected movement and display the 3D model of thefirst segment at the updated orientation.
 10. The system of claim 9,wherein the fifth user input for entering the mode for rotating thedisplayed 3D model of the first segment using the inertial measurementdata from the inertial measurement device of the hand held intraoralscanning device is a hold gesture.
 11. The system of claim 8, whereinthe first user input and the second user input are touch gestures. 12.The system of claim 8, wherein at least one of a) the hand heldintraoral scanning device is further to or b) the instructions furthercause the processor to: determine that the first user input comprises afirst gesture; and determine that the second user input comprises asecond gesture.
 13. The system of claim 8, wherein: the one or morebuttons are to receive the fourth user input for navigating to theprevious segment; and responsive to the intraoral scanning devicereceiving the fourth user input, the hand held intraoral scanning deviceis to enter the scan mode to begin rescanning the first segment of thepatient's oral cavity.
 14. The system of claim 13, wherein responsive tothe hand held intraoral scanning device rescanning the first segment ofthe patient's oral cavity, the instructions cause the processor to:receive one or more additional intraoral scans of the first segment fromthe hand held intraoral scanning device; and revise the 3D model of thefirst segment based on the one or more additional intraoral scans of thefirst segment.
 15. A method comprising: receiving a first user input viaa hand held intraoral scanning device; responsive to receiving the firstuser input, causing the hand held intraoral scanning device to enter ascan mode; responsive to the hand held intraoral scanning devicescanning a dentition in an oral cavity of a patient, receiving one ormore intraoral scans of the dentition; generating a three-dimensional(3D) model of the dentition based on the one or more intraoral scans;displaying the 3D model of the dentition to a display; receiving asecond user input via the hand held intraoral scanning device afterreceiving one or more intraoral scans of the dentition; responsive toreceiving the second user input, displaying to the display an overlay onthe displayed 3D model of the dentition, the overlay including a firstelement corresponding to third user input for navigating to a nextsegment, a second element corresponding to fourth user input fornavigating to a previous segment, and a third element corresponding tofifth user input for entering a mode for rotating the displayed 3D modelof the dentition using data from an inertial measurement device of thehand held intraoral scanning device; receiving, after displaying theoverlay on the displayed 3D model of the dentition, the fifth user inputfor the mode for rotating the displayed 3D model of the dentition usingdata from the inertial measurement device of the hand held intraoralscanning device; responsive to receiving the fifth user input for themode for rotating the displayed 3D model of the dentition using datafrom the inertial measurement device of the hand held intraoral scanningdevice, detecting a movement of the hand held intraoral scanning devicevia the inertial measurement device of the handheld intraoral scanner;and responsive to detecting the movement, performing the followingcomprising: updating an orientation of the 3D model of the dentitionbased on the detected movement; and displaying the 3D model of thedentition at the updated orientation.
 16. The method of claim 15,wherein the fifth user input for entering the mode for rotating thedisplayed 3D model of the dentition using the data from the inertialmeasurement device of the hand held intraoral scanning device is a holdgesture.
 17. The method of claim 15 further comprising: determining thatthe first user input comprises a first gesture; and determining that thethird user input comprises a third gesture.
 18. The method of claim 15further comprising: receiving the third user input via the hand heldintraoral scanning device; and responsive to receiving the third userinput, controlling the user interface overlaid on the 3D model of thedentition to navigate to the next segment.
 19. The method of claim 18,wherein the first user input, the second user input, and the third userinput are touch gestures.
 20. A method for scanning multiple intraoralsegments of a patient's oral cavity, the method comprising: displayingto a display a first indication of a first segment of the patient's oralcavity to be scanned; receiving a first user input via a hand heldintraoral scanning device; responsive to receiving the first user input,causing the hand held intraoral scanning device to enter a scan mode;responsive to the hand held intraoral scanning device scanning the firstsegment of the patient's oral cavity, receiving intraoral scans of thefirst segment; generating a three-dimensional (3D) model of the firstsegment based on the one or more intraoral scans of the first segment;displaying to the display the 3D model of the first segment; receiving asecond user input via the hand held intraoral scanning device afterreceiving intraoral scans of the first segment; responsive to receivingthe second user input, displaying to the display an overlay on thedisplayed 3D model of the first segment, the overlay including a firstelement corresponding to third user input for navigating to a nextsegment and a second element corresponding to fourth user input fornavigating to a previous segment; receiving, after displaying theoverlay on the displayed 3D model of the first segment, the third userinput for navigating to the next segment via the hand held intraoralscanning device; entering the scan mode after receiving the third userinput for navigating to the next segment; responsive to the hand heldintraoral scanning device scanning the second segment of the patient'soral cavity, receiving one or more intraoral scans of the secondsegment; generating a 3D model of the second segment based on the one ormore intraoral scans of the second segment; displaying to the displaythe 3D model of the second segment; receiving the second user input viathe hand held intraoral scanning device after receiving intraoral scansof the second segment; responsive to receiving the second user inputafter receiving intraoral scans of the second segment, displaying to thedisplay the overlay on the displayed 3D model of the second segment, theoverlay including the first element corresponding to the third userinput for navigating to the a next segment and the second elementcorresponding to the fourth user input for navigating to the previoussegment; receiving, after displaying the overlay on the displayed 3Dmodel of the second segment, the fourth user input for navigating to theprevious segment via the hand held intraoral scanning device; enteringthe scan mode after receiving the fourth user input for navigating tothe previous segment; responsive to the hand held intraoral scanningdevice rescanning the first segment of the patient's oral cavity,receiving one or more additional intraoral scans of the first segment;and revising the three-dimensional (3D) model of the first segment basedon the additional intraoral scans of the first segment.